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Abstract. Primarily motivated by the stability analysis of nonlinear waves in second-order in time Hamil- 
tonian systems, in this paper we develop an instability index theory for quadratic operator pencils acting 
on a Hilbert space. In an extension of the known theory for linear pencils, explicit connections are made 
between the number of eigenvalues of a given quadratic operator pencil with positive real parts to spectral 
information about the individual operators comprising the coefficients of the spectral parameter in the 
pencil. As an application, we apply the general theory developed here to yield spectral and nonlinear sta- 
bility/instability results for abstract second-order in time wave equations. More specifically, we consider 
the problem of the existence and stability of spatially periodic waves for the "good" Boussinesq equation. 
In the analysis our instability index theory provides an explicit, and somewhat surprising, connection be- 
tween the stability of a given periodic traveling wave solution of the "good" Boussinesq equation and the 
stability of the same periodic profile, but with different wavespeed, in the nonlinear dynamics of a related 
generalized Korteweg-de Vries equation. 
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1 . Introduction 

When analyzing equations arising in mathematical physics and engineering, the question of stability of 
special families of solutions is of prominent importance as it generally determines those solutions which 
are most likely to be observed in physical applications. In particular, solutions which are unstable do 
not naturally, i.e. in absence of a controller, arise in applications except possibly as transient phenomena. 
Furthermore, stability analysis is often the first step in the study of finer phenomena such as transient 
behavior, bifurcation, and the ability to control a wave to restrict it to a stable configuration. In this paper, 
we are primarily motivated by recent studies into the stability of traveling wave solutions of second-order 
in time Hamiltonian equations of the form 

d^u + C^u + N{u) = 0, {t,x)elR^ (1.1) 

where Cx is a self-adjoint linear operator acting on the x-variable only, and Af{u) denotes nonlinear terms 
(e.g., see [2, 3, 15, 34]). A fundamental characteristic of such PDE is that they take into account weak 
effects of both nonlinearity and dispersion, and they arise naturally, for instance, as models for propaga- 
tion of waves in nonlinear strings and in the study of bi-directional water wave propagation in the small 
amplitude, long wavelength regime. 

In regards to the latter water wave application, an equation of particular interest in this paper (see 
Section 4.2 below) is the generalized "good" Boussinesq (gB) equation 

dju-dl{dlu-u + f{u)) = 0, (1.2) 

which is a variant of one of one of the equations formulated by Boussinesq in the 1870's in precisely this 
physical context. While the nonlinear stability and instability of solitary waves in (1.2) is by now well 
understood (see [4, 28]), the stability (whether linear or nonlinear) of the periodic traveling waves have 
received considerably less attention, and results only exist for very special classes of solutions; namely, 
those expressible in terms of Jacobi-elliptic functions. One of the main applications of the theory developed 
in this paper is an explicit connection between periodic traveling waves of (1.2) and the stability of the 
same traveling wave profile (but with a different wavespeed) in the nonlinear dynamics governed by the 
generalized Korteweg-de Vries (gKdV) equation 

dtu + dx[dlu+ f(u)) = Q. (1.3) 

We consider this observation as a major contribution to the theory of traveling periodic waves in (1.2). 
Indeed, since the stability of periodic traveling waves in gKdV equations has been under intense investiga- 
tion over the last few years (see, for instance, [1, 3, 5, 6, 8, 9, 18]), this connection between the dynamics 
of gB and gKdV near periodic traveling waves allows one to immediately translate known results for the 
stability of gKdV periodic waves to results about periodic waves in gB. Applications of this connection will 
be illustrated in Section 4.2.1 and Section 4.2.2 below. 

Suppose that u{x-ct) is a traveling wave solution to (1.1). When considering small perturbations to this 
wave of the form e^*v{x - ct), A e C, we are naturally led to quadratic spectral problems of the form 

X^v - IcXd^v + [c^dl + Lx + N\u))v = 0. 

More generally, when considering the spectral stability of the given nonlinear dispersive wave u it becomes 
important to understand the spectrum of quadratic operator pencils of the form 

V2{X):=A+XB + X^C, (1.4) 

where the operators A and C are self-adjoint on some Hilbert space X, endowed with an inner-product {•, •), 
and B is skew-symmetricon X. The class of perturbations considered in our applications naturally leads us 
to assume that the domain of each of the operators A, B, C is dense in X and that they each enjoy a particular 
compactness property which guarantees that the spectrum of 7-'2(A), i.e., the collection of values A for which 
V2{X) fails to be boundedly invertible, is composed of point spectrum only, that each eigenvalue has finite 
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algebraic multiplicity, and the only accumulation point of the eigenvalues is infinity (see Lemma 2.1 below 
for a precise statement). In this paper a{V2) will denote the collection of all eigenvalues for the pencil. 

Due to its clear connection and importance in analyzing the spectral stability of traveling wave solutions 
of equations of the form (1.1), our main theoretical results concern extending many previously known 
results regarding the number of eigenvalues of V2 with positive real part. For the readers convenience, 
we now briefly recall the relevant known results. The spectrum was studied in [32] under the assumption 
that A has compact resolvent, and B,C are bounded and positive semi-definite. The operators B,C are not 
assumed to have any symmetry properties, however. While it is not shown in that paper, by Lemma 2. 1 it is 
then known that the pencil only possesses point eigenvalues, each of which has finite algebraic multiplicity. 
It is shown in that paper that if A is positive semi-definite, then all of the spectrum is located in the closed 
left-half of the complex plane. If A is not definite, it is shown that if C = I and B is positive definite, then 
the total number of eigenvalues in the closed right-half of the complex-plane is equal to the number of 
negative eigenvalues of A. As is seen in, e.g.. Section 3, it is not necessary that C = I in order for that result 
to hold; indeed, all that is needed is that C be positive-definite. In [29] similar results are shown regarding 
the number of eigenvalues with positive real part under the assumption that C = I and B is positive semi- 
definite. The spectrum of operators of the form Vi was studied in [14] under the assumptions that both 
A, C are positive definite and self-adjoint, while ;8 is a negative definite self-adjoint operator (also see [26] 
for a generalization). Under assumptions different than those given in Lemma 2.1 (in particular, both A 
and C are assumed to be compact) it is shown therein that in 0(7^2) there is an infinite number of positive 
eigenvalues which have zero as a limit point. Finally, the matrix -valued version of the pencil was studied 
in [7, 26] under the assumption that C = X and A, B self-adjoint. Therein a parity index is given relating 
the number of eigenvalues with positive real part to the number of negative eigenvalues of A. This result 
strongly depends upon the fact that the operators are matrix-valued, and consequently the Hilbert space 
under question is finite-dimensional. 

One of the main goals of this paper is thus to extend the previous theory regarding the number of 
eigenvalues (counting multiplicity) with positive real part; see Section 3 below. Throughout this analysis, 
two underlying assumptions will be: 

(a) n(^),n(C) < +00, where n(S) refers to the number of negative eigenvalues (counting multiplicity) 
of the self-adjoint operator S 

(b) C is invertible, i.e., z(C) = 0, where z{S) = dim[ker(5)] for the self-adjoint operator S. 

Under these assumptions we have the intuition that the sum n(^) -1- n(C) acts as an upper bound on the 
total number of eigenvalues of V2 with positive real part. In order to see this, consider the following. If the 
linear term in the pencil is dropped, then the quadratic pencil is equivalent to the linear pencil 

Note that the first term in the pencil is self-adjoint, whereas the second term is skew-symmetric. Linear 
pencils of this form have been well-studied (e.g., see [17, 24, 25, 31] and the references therein), and for 
this problem it has firmly been established that the intuition is indeed correct. The technical difficulty is 
then the inclusion of the linear term, and it is overcome in the establishment of Theorem 3.1 below. In 
particular, in the case where ViiX) arises in the stability analysis of a given periodic traveling wave (as 
described previously), a sufficient condition for spectral stability is that n(^) -1- n(C) = 0. 

In order to achieve an equality relating the number of eigenvalues of V2 with positive real part to spec- 
tral properties of the operators A, B, and C themselves, two additional factors must be taken into account: 

(a) the effect of A having a nontrivial kernel 

(b) purely imaginary eigenvalues having a negative Krein index (see Section 3). 

The first factor arises because in applications the presence of symmetries yields the existence of a nontrivial 
kernel, while the consideration of the second factor is necessary in order to remove the intuitive inequality 
and make it an equality. 
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The paper is organized as follows. In Section 2 it is shown that under general assumptions, which are 
natural for the applications we have in mind, there is no essential spectrum for the quadratic operator pen- 
cil Vi', in other words, there will be only point eigenvalues, and each eigenvalue will have finite algebraic 
multiplicity. The proof of this result easily generalizes to polynomial operators of arbitrary order; see Re- 
mark 2.2 below. In Section 3 the main theoretical result of the paper relating the number of eigenvalues 
of Vi with positive real part to the spectral properties of A, B, and C, are stated and proved. Finally, in 
Section 4 we apply the theoretical results developed in Section 2 and Section 3 to the study of the spectral 
and orbital stability of nonlinear waves to second-order in time Hamiltonian systems. The first application 
develops a general theoretical result concerning the stability of steady states in abstract nonlinear wave 
equations; see Section 4.1. The second application examines the spectral and nonlinear (orbital) stability 
of periodic traveling waves in the "good" Boussinesq equation (1.2); see Section 4.2. As described above, 
of particular interest in our study of the Boussinesq equation is that we establish in Lemma 4.2 and Theo- 
rem 4.8 a rigorous connection between a given stationary periodic wave u of the equation 

d]u - Icdj^u + d\ [d\u - (1 - c^)u + f{u)) = 0, \c\ < 1, 

corresponding to the traveling wave u{x-ct) in the gB, and the stability of the same stationary wave profile 
in the gKdV, 

dfU + [dl - Vl-c2 u + f{u)) = 0. 

To illustrate the power of this connection, in Section 4.2.1 we translate recent results by Bronski et al. [6] 
concerning the stability of periodic KdV waves with power-law nonlinearity to results for the corresponding 
Boussinesq equation, allowing one to see the complete stability picture of periodic waves in gB in the cases 
considered. In Section 4.2.2, using known asymptotic analyses worked out in the context of stability theory 
for gKdV waves (see [5]), we analyze the stability of periodic waves with power-law nonlinearity which 
are near (in an appropriate sense) either the solitary wave or equilibrium solutions of gB. The analysis 
near the solitary wave makes a beautiful connection with the known nonlinear stability/instability results 
for the nearby solitary waves. Similarly, the analysis near the equilibrium solution provides new insights 
into the transitions to instability in the periodic context; in particular, we find that for a given power-law 
nonlinearity stable periodic traveling wave solutions always exist, even when no stable solitary wave exists. 

Acknowledgments. TK gratefully acknowledges the support of the Jack and Lois Kuipers Applied Math- 
ematics Endowment, a Calvin Research Fellowship, and the National Science Foundation under grants 
DMS-0806636 and DMS-1 108783. MJ was partially supported by the University of Kansas General Re- 
search Fund allocation 2302278. JB gratefully acknowledges support from the National Science Foundation 
under grant DMS-DMS-0807584. 

2. Preliminary result: spectra is point only 

Notation: In this paper, and particularly in this and the subsequent section, the notion of matrix represen- 
tation of a self-adjoint operator S constrained to a subspace E, i.e., Slg, will often be used. Let {ei,...,e„} 
be a basis for E, and let : X £ be the orthogonal projection. If the basis is orthonormal, then one can 
write 

n 

i=i 

Setting S\e = PeSPe : E E, the quadratic form 

{u,S\eu) = {Peu,PeSPeu) = c - Sc, 

where PeU = J^CjCj, c = (cj,. . .,c„)^, and the symmetric matrix S e ]R"^" is given by S,y = {ei,Sej). The matrix 
S is precisely the matrix representation for the self-adjoint operator S\e- In the rest of this paper the symbol 
S\e will be used to represent the matrix representation S of 5 constrained to operate on the subspace E. 
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The goal of this section is to demonstrate that for the quadratic pencil of (1.4), i.e., 

r2{A) = A + XB + A^C, 

there is point spectra only, that each eigenvalue has finite multiplicity, and infinity is the only possible limit 
point of the eigenvalues. The result requires the use of [30, Theorem 12.9], in which it is shown that the 
spectrum has the desired properties for the polynomial operator 

77 

;=1 

if each operator Aj, ; = is compact. With this result in mind, first assume that A is invertible, and 

rewrite the eigenvalue problem as 

A(I + XA-^B + X^A-^Qu = 0. 

Of course, since A is nonsingular this problem is equivalent to 

(I + XA-^B + X^A-^C)u = Q. 

If the operators A~^B and A~^C are both compact, then the result immediately follows from [30, Theo- 
rem 12.9]. 

On the other hand, now suppose that ker(^) is nontrivial, but that A has compact resolvent. The proof 
of the desired result will now be accomplished via the construction and evaluation of a generalized Krein 
matrix, which was recently introduced in a general form in [21]. Let P4 : X ker(^) be the orthogonal 
projection. Writing u = a + a-*-, where a e ker(^) and e ker(^)^, the eigenvalue problem becomes 

7'2(A)fl + P2(A)fl-L = 0. (2.1) 

Defining the complementary projection := I - P4, applying this projection to (2.1), and solving for 

fl-*- = P^a-^ yields 

«^ = -(Pt7'2(A)P|)-ip|7'2(A)fl. (2.2) 

In the formulation of (2.2) it is implicitly being assumed that V2{X)a^ ^ 0. If V2{X)a^ = 0, then A is an 
eigenvalue whose eigenfunction is in ker(^)-'-: this follows immediately from (2.1) upon setting a = 0. 
Since 7-'2(A)fl = 0, the potential pole singularity for such a A is removable. If the inner-product with a is now 
taken in (2.1), then it is seen that 

{a,r2{X)a) + {a-^,r2{Xfa) = 0, (2.3) 

where 

ViiXf =A-XB + X^C 

is the adjoint operator for the original pencil. Note that the fact that A,C are self-adjoint and B is skew- 
symmetric was used in this formulation of the adjoint pencil. Substituting the expression for s-*- given in 
(2.2) into (2.3) yields the linear system 

K2{X)X = 0, K2{X) := 7'2(A)|kerM) - (J'i7'2(A)P|)-l|pXp^(;^)[i,,,(_4)]. (2.4) 

Here 

X e C"*-^', K2(X) e C^*'^)^^^*'^), 

and 

7'2(A)[ker(y4)] := |7'2(A)a : a e ker{A)}. 

The matrix K'2(A) is known as the Krein matrix. 

Eigenvalues for the pencil are found either via x = 0, which means that if A is an eigenvalue, then the 
associated eigenfunction satisfies u e ker(^)-'-, or det[K'2(A)] = 0. The quest for eigenvalues is now a search 
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for the zeros of the determinant of the Krein matrix. The poles of the Krein matrix are the eigenvalues of 
the operator 

Pa'^iWPa = Pa^Pa + ^Pa^Pa + ^^Pa^Pa ■ kerly^l)-^ ^ kev(A)^. 
Since P^AP^ : ker(^)-'- ker(^)-'- is invertible, from the earlier argument in this section we know that 
the operator P^'P2('^)Pa point spectra only, each eigenvalue has finite multiplicity, and infinity is the 
only possible limit point of the eigenvalues. Thus, one can say that det[iiC2(A)] : C i-^ C is meromorphic, 
each singularity is a pole of finite order, and the only possible accumulation point of the poles is infinity. 
Since det[K2(A)] is meromorphic, it is then known that each of its zeros is of finite order, and that the only 
possible accumulation point of the zeros is infinity. Finally, it was demonstrated in [21] that for linear 
pencils the order of the zero of det[iiC2('^)] is equal to the algebraic multiplicity of the eigenvalue. The proof 
of that result easily carries over to quadratic pencils, and will be left for the interested reader. 

Lemma 2.1. Let P4 : X ker(^) be the orthogonal projection, and let P^ = I - Pa be the complementary 
projection. Suppose that the operators 

(P|^P|)-ip|SP|, (P^AP^r^P^CP^ ■ ker(^)^ ^ ker(A)^ 

are both compact. Then the spectrum of the quadratic pencil A + XB + X^C is point spectra only. Furthermore, each 
eigenvalue has finite multiplicity, and infinity is the only possible limit point of the eigenvalues. 

Remark 2.2. Note that the proof of Lemma 2.1 did not require that any of the operators have a symmetry 
property. Thus, this lemma can be thought of as a general result about quadratic pencils. Indeed, although 
it will not be proven here, the above argument can be extended to show that for -order polynomial 
pencils of the form 

n 

Vn(X) = J^XiAj, 

i=o 

if each of the operators {P^^AqP^^)~^ P^^AjP^^ is compact for = l,...,n, then the spectrum for this pencil 
will have exactly the same properties as that for the quadratic pencil. 



3. Main result: the instability index theorem 



The goal here is to derive an instability index theorem for the quadratic pencil. In applications it may be the 
case that is bounded and not compact, and/or B is not bounded. In order to overcome these technical 
difficulties (which are associated with the proof only), take a positive definite self-adjoint operator S which 
has a compact inverse, and consider the quadratic pencil 

V2W := A+ XB + X^C. 
The operators in this new pencil are related to those of the original pencil by 

A:=S-'^AS-\ B:=S-^BS-\ C:=S-^CS-\ 

so that 

p2(A) = 5^2(A)5. 

The working assumption is that B, C are 5-compact, while A has 5-compact resolvent, where an operator 
T is said to be 5-compact if the operator S~^TS~^ is compact, and is said to have 5-compact resolvent if 
the inverse ST~^S (defined to act on the range space) is compact. 
Define the space Xg by 

:= {u eX: {Su,Su) < 00). 

It will be assumed that Xg C X is dense. Since is compact, and therefore bounded, it is clear that 
o{p2) ^ o(V2) on X. It is also clear that oiVi) when considered on the space Xg is a subset of a(V2) when 
considered on the space X. In other words, it is true that 



ff (7^2) on Xg C a(V2) on X C 0(^2) on X. 
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Since Xg C X is dense, by [33, Proposition 3.2] it will be the case that 

o{P2) on Xs = a{V2) on X. 

Finally, since is bounded, it is true that n(^) = n(A) and n(C) = n(C). In conclusion, from this point 
forward the "hat"'s associated with the operators can be dropped, and it will be assumed that the operators 
satisfy C, B are compact. 

The instability index theorem for the quadratic pencil will be proven by constructing an equivalent 
linear pencil, and then deriving an index theorem for that linear pencil. Upon setting w = {u, ACu)^, the 
quadratic pencil (1.4) is linearized to become 

(C-AJ-^)w = 0, (3.1) 

where 

A \ j^l I 
C-i j' ^ \ -I -B 

Here is where the assumption that C be invertible comes into play. Since B is skew-symmetric, (3.1), and 
consequently the pencil (1.4), is formally equivalent to the Hamiltonian eigenvalue problem 

J£w = Aw, (3.2) 

where J' is skew-symmetric and £ is self-adjoint. Note that in this formulation 

(a) £ has compact resolvent 

(b) has bounded inverse 

(c) the spectrum satisfies the symmetry |A,-A) C a(JC), and if all of the operators have zero imagi- 
nary part, the symmetry becomes {+A,+A} C a(JC). 

Since C is invertible, it is clear that for nonzero A the two eigenvalue problems have identical eigenval- 
ues; furthermore, the geometric multiplicities match. As it will be seen, it is also the case that the algebraic 
multiplicities of these eigenvalues also coincide. We will show that this is true at the origin: the proof will 
clearly generalize to the case of a nonzero eigenvalue. We must consider the structure of gker( J^/I) and the 
manner in which it relates to gker(7-'2(0))- 

First consider gker(7'2(0)). It is clear that ker(7'2(0)) = ker(^). Following Markus [30], generalized 
eigenfunctions are found by solving 

V2(Q)ai+V2{Q)aQ = Q, aq e ker(y4). 

This is equivalent to solving 

Aui = -Buq, (3.3) 

which by the Fredholm alternative has a nontrivial solution if and only if Buq e ker(.4)-'-. Since B is skew- 
symmetric, it is not unreasonable to assume that '8|i;er(^) = 0. In this case there is a solution to (3.3) for any 
flo € ker(^), and a full set of associated eigenfunctions is given by ^~^,8ker(^), where the obvious notation 

A-^Bker(A) := {A'^Ba : a e ker(A)] 

is being used. The next set of eigenfunctions is found by solving 

P2(0)fl2 + 'PiWai + ^7'2'(0)ao = 0. ^0 e ker(A), fli = -A'^Buq e A~^Bker{A). 
This equation is equivalent to 

Aa2=-Bai-Cao = -{C-BA-^B)ao. (3.4) 

Again using the Fredholm alternative, it is seen that there is a nontrivial solution to (3.4) if and only if 
(BA-'^B-C)aoeker(A)'^. Upon using the fact that for any a e ker(^), 

{B(A-^Bao),a) = -{A'^ Bao,Ba), 
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it is seen that if 

D:= (C-S^-lS)lkerM) 

IS nonsme ular, then (C - BA-^B)ao £ ker{A)-^ for any Aq ^ ker(^). It will henceforth be assumed that D is 
nonsingular. 

Now consider gker(J'£). It is clear that ker(£) = (ker(^), 0)^. Since 

J-^ ker(£) = II I : flo e ker^jj , (3.5) 

upon writing lu = {u,v)^ the generalized eigenfunctions are found by solving 

Au=-BaQ, C~^v = aQ. 

The first equation is precisely (3.3), which was seen to have a solution for any Uq e ker(^). Consequently, 
there is a generalized eigenspace at A = which is given by {(-A-^BaQ,Caof : uq e ker(y4)). Since 



J- 



i( -A-^Bao \_ I (C-BA-^B)ao 



the next set of generalized eigenfunctions is found by solving 

Au=-(C-BA-^B)ao, C'^v = -A'^BaQ. 

The first equation is precisely (3.4), which was seen to have no solution under the assumption that D is 
nonsingular. 

It is now seen that regarding the algebraic multiplicity of the eigenvalue the Hamiltonian linearization 
(3.2) is equivalent to the pencil at A = 0. Following the same argument for nonzero A it is not difficult to 
check that this equivalence continues to hold; namely, the location of the eigenvalues, and their multiplici- 
ties, are the same for the two systems. 

We are now ready to derive the instability index for the linear Hamiltonian system (3.2). We must first 
construct the appropriate closed subspace on which both £ are nonsingular. Recalling that : X 
ker(^)-'- is the orthogonal projection, let : XxX ker(.4)-'- x X = ker(£)-'- be given by 



£ ■ \ I 

in other words, for w = {u,v) e X x X it is true that Tl-^w = {P^u,v). Define another orthogonal projection 
by 

n J-> ker(£) : X X X [ keriC)]^. (3.6) 

Because ker(£) ± J'"^ker(£), the projections ^c'^j-i]fieT{C) commute. Upon setting n := ^c^j-i]!^ei{C) 
(note that Tl being the composition of self-adjoint commuting operators implies that it too is self-adjoint), 
nonzero eigenvalues for the linearization (3.2) are found by solving 

njn-ncn-nw = XUw, nu;e[ker(C)(Bj~'^ker{C)]-^ (3.7) 

(e.g., see [8, Section 2]). This is the eigenvalue problem to be studied in the rest of this section. 

The goal is to now count the total number of eigenvalues in the open right-half of the complex plane 
(counting multiplicity), along with a those eigenvalues on the imaginary axis which have negative Krein 
index. For each A e ilR let E;^ denote the generalized eigenspace. The negative Krein index of the eigenvalue 
for the linearized system (3.2) is given by 

k[(X) :=n(C\E,), 

and if k^(X) = 0, then the eigenvalue is (often) said to have positive Krein signature. Let us now relate this 
definition to a definition using the quadratic pencil. By the definition of w leading to the linearization (3.1) 
one has that 

£\e, = (A + \X\'C)\n,E, = {A- X'C)\n,E,, 
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where Yl^ : XxX X is the projection onto the first component, i.e., ni(M, v)^ = u, and the second equality 
follows from the fact that A 6 ilR. Now, A being an eigenvalue with associated eigenfunction u means that 

r2{A)u = =^ Au=-XBu-A^Cu, 

which in turn implies 

(-4-A2c)|n,£,=-AP^(A)|n,£,. 
In conclusion, the negative Krein index for the quadratic pencil is defined to be 

/cr(A):=n(-A7'^(A)|£j, (3.8) 

where now E;^ = gker(7'2(A)), i.e., the generalized eigenspace of the quadratic pencil 7'2('^) associated with 
the eigenvalue A e ilR. 

We are now ready to derive the index formula. For the eigenvalue problem (3.7) let represent the 
number of positive real-valued eigenvalues (counting multiplicity), and let be the number of complex- 
valued eigenvalues (counting multiplicity) with positive real part. Furthermore, let the total negative Krein 
index be given by 

kr := ^ kr(X). 
CT(P2(A))ni]R 

Regarding the eigenvalue problem (3.7) it is known that (n£n)~^ is compact, and that the operator n^/n 
is bounded with bounded inverse. Indeed, since one can write 

njn = nl i Mn + n( « Ma 



I I \ -B 

one actually has that both H JTI and (H JTI)~^ can be written as the sum of a bounded operator and a com- 
pact operator. Thus, upon using the fact that compact operators are uniformly approximated by matrices, 
when computing an index which takes into the account the (finite) number of negative directions of an 
operator it is sufficient to consider the case of matrices only. For the eigenvalue problem (3.7) when the 
operators are matrices it is known from [17] that 

k, + k^ + kr = n(ncn). (3.9) 

Since all of the quantities are integer-valued, by taking the limit one deduces that the result holds for the 
full operators. 

Before stating the final result, the quantity n(n£n) must be computed in terms of the original operator 
£. It is known (e.g., see [23, Index Theorem]) that 

n{UCU) = n{C) - n(£-i j ^-i ker(£)). (3.10) 

It is clearly the case that 

n(C) = n(A) + n(C). 
Furthermore, it is straightforward to verify that 

^~^\j-^ker(C) = (C-S-4"^B)|ker(yt)- 

The instability index (3.9) can then be rewritten as 

n(n/:n) = n(^) + n(C) - n((C - S^-iS)|ker(^)). (3.11) 
Combining (3.9) with (3.11) yields the following theorem: 

Theorem 3.1. Suppose that the operators satisfy the assumption of Lemma 2.1. Further suppose that there is a 
self-adjoint and positive operator S such that 

(a) Xg := {u eX : {Su,Su) < oo) c X is dense 
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(b) the operators B,C are S-compact, and the operator A has an S-compact resolvent. 
Finally, assume that the operator C is invertible. If 

(i) ^lker(^) = 0^(A), and 

(ii) {C - BA~^B)\]^g^^J^) is invertible, 

then the total number of eigenvalues in the closed right-half of the complex plane satisfies the instability index 

K + k, + k- = n{A) + n(C) - n((C - BA-' B)]^,,,^^^). 

Remark 3.2. If A is nonsingular, then Theorem 3.1 is precisely the result of [33, Corollary 3.9]. On the other 
hand, in the event that A has a nontrivial kernel then it is an improvement over [33, Theorem 4.2], where 
it was shown that 

k, + kc + k[ <n{A) + n{C). 

The proof presented here is quite different than that given in [33]; in particular, in that paper the anal- 
ysis takes place on Pontryagin spaces, and the linearization that is studied there is not the Hamiltonian 
linearization of (3.2). 

Remark 3.3. If the imaginary part of all of the operators is zero, then due to the Hamiltonian eigenvalue 
symmetry {+A, +A) C ct(7-'2) it is necessarily the case that k^ and fcr are even-value. Thus, under this assump- 
tion n(n£n) being odd automatically implies that k^>l. On the other hand, if one or more of the operators 
has a nontrivial imaginary part, then the Hamiltonian eigenvalue symmetry reduces to {A, -A) C a(P2), and 
no such conclusion can be drawn. 

Remark 3.4. One consequence of the index is that all but a finite number of eigenvalues are purely imagi- 
nary; furthermore, the purely imaginary eigenvalues have positive Krein signature if the modulus is suffi- 
ciently large. 

Remark 3.5. The proof of the index formula (3.9) in [17] first required the SCS Basis Lemma, in which it was 
shown that the generalized eigenvectors associated with the linearization (3.1) formed a basis. Technical 
assumptions on the operators were needed in order for the SCS Basis Lemma to hold true. Unfortunately, 
at least in the application discussed later in this paper these technical assumptions do not hold; hence, the 
alternate proof via the limiting argument. 

4. Applications to second-order in time Hamiltonian systems 

As discussed in the introduction, quadratic operator pencils arise naturally in when one studies the stability 
of solutions to second-order in time Hamiltonian systems. In this section, we present two applications of 
the general theory developed in the previous sections in precisely this context. We begin by considering the 
stability of periodic waves in an (abstract) nonlinear wave equation posed in a Hilbert space, and conclude 
with a stability analysis for periodic waves in the so-called "good" Boussinesq equation. 

4.1. Example: stability in (abstract) nonlinear wave equations 

One important example of quadratic pencils arises in the study of second-order (in time) Hamiltonian 
systems (for a specific case of the following discussion, see, e.g., [13, Section 7]). Consider a wave equation 
of the form ^ 

dju + n'(u) = 0, ?^W("> := ^(w). (4.1) 

du'^ 

where u e X, which is a Hilbert space with inner-product (•,•). The Hamiltonian H : X h^Mis assumed to 
be smooth. 

It will be assumed that the Hamiltonian system has symmetries. Let G be a finite-dimensional abelian 
Lie group with Lie algebra g. Denote by exp(w) = e'" for w e g the exponential map from g into G, and 
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assume that T : G HV), where X C V C X' (the dual space of X), is a unitary representation of G on V. 
It is then the case that T'{e) maps g into the space of closed skew-symmetric operators on V with domain 
X. The notation 7^ := T'{e)a> for co e g will be used to denote the linear skew-symmetric operator which 
is the generator of the semigroup T{e"^). Using this notation the symmetry assumption becomes that the 
Hamiltonian satisfies T(co)Ti.(u) = 'H{T{co)u) for all a; e g. 

Writing u = {u,vf- , where v = dfU e Xj (in applications, it is often the case that X c Xj is dense), the 
system (4.1) can be written on X x X^ as the first-order Hamiltonian system 

dtu = jn'(u), (4.2) 

where 

^ = (-1 )' nu) = n(u)+\{v,v). 
The system (4.2) is invariant under the action T(u)), where 

T{uj) 



T{u:)u ■■ 



Tiu) 



An n-parameter family of conserved quantities for the Hamiltonian system (4.2) is induced from the self- 
adjoint operator J T^^, and is given by 

Q(m) := \ {J-'f^u, m) = - Re {{T^u, v)). 

Upon defining the Lagrangian 

A(m) ■.= n{u) + Q{u), 
waves to (4.2) will be realized as steady-state solutions for the system 

dtU = JA'(u), (4.3) 

i.e., they are critical points for the Lagrangian. Since 

A>) = W + J-^r„(u) = (^'("J;J'''^ 

critical points are solutions to 

?^'(M) + rjM = 0, T;^,u:=T^{T^u). (4.4) 
It should be noted here that (4.3) is equivalent to the second-order problem 

d\u + 2T^dtU + n'(u) + T^,u = 0. (4.5) 

Suppose that m = 1/ is a solution to (4.4) (the w-dependence of the solution is being suppressed here), so 
that U = (U ,T^UY is a critical point of the Lagrangian. Indeed, further suppose that there is a nonempty 
open set Q c g such that the solution is smooth in w for all w e Q, and further assume that the isotropy 
subgroups e G : T{g)U = U] are discrete for all a>. Now consider the spectral and orbital stability of the 
wave. The linearized problem associated with (4.3) is given by 

dtU = JCu, (4.6) 

where the self-adjoint operator £ is 

The eigenvalue problem for (4.6) is given by 

JCu = Am. 
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This eigenvalue problem is the system 

-T-l"{U)u-T^^,v = Av, -Tf^u + v = Xu, 
which after substitution is equivalent to the quadratic pencil 

(n"(U) + Ti^,+2XT^ + X^I)u = 0. (4.7) 

In the notation of (1.4) one has 

A = U"{4>) + T^,, B = 2T^„ C=I. 

Note that the operators A, C are self-adjoint, while the operator B is skew-symmetric. It is interesting to 
note that the negative index of £ is discussed in [27, Lemma 1], where it is stated that 

n(/:) = n(7i"(</>) + Tj). 

The number of negative directions of C is precisely the number of negative directions associated with the 
linearization of (4.4) about u = U. 

With respect to the spectrum of the pencil (4.7) the result of Theorem 3.1 says the following. The 
assumptions associated with the symmetries present in the problem imply that 

ker(?^"(L/) + T,?) = span{T^[/), 

so that z(H"(U) + T^) = n. Furthermore, these assumptions imply that 

T;, : ker(?^"( U) + TJ) ker(?^"( U) + TJ)^, 

so that the generalized kernel for the pencil has (at least) dimension In. Since C - X, under the assumption 
that the matrix 

(I - ^T^{U"{1]) + Tj)-iT„)|,panK,[/) 
is invertible, it will then be the case that the instability index count satisfies 

k, + k, + k- = n(n"{4>) + Ti) - n((I - 4Tjn"(U) + T,^)-'TJ\,p,„^r^^u\) (4.8) 

Now that the spectral problem is understood, consider the orbital stability of the wave. This result 
follows almost immediately for [13, Theorem 4.1]. An alternate interpretation of that result is as follows. 
In the language of that paper the wave is said to be orbitally stable if the reduced Hamiltonian, which is 
the Hamiltonian restricted to the closed subspace orthogonal to the generalized kernel of JJC, is positive 
definite. As was discussed in, e.g., [8, 9, 20], this condition is equivalent to saying that for the linearized 
problem (4.6) the spectrum is purely imaginary and satisfies fcr = 0. Under this spectral assumption, and 
the compactness assumptions associated with the operators, the wave is then a local minimizer for the 
Lagrangian, and hence is orbitally stable. 

Theorem 4.1. Suppose that for the quadratic pencil 

P2W := {H"{U) + rj) + X{2TJ + X^I, 

which is the spectral problem for the linearization of the second-order Hamiltonian system (4.5) about the steady- 
state u = U, the operators satisfy the assumptions associated with Theorem 3.1. Assume that solutions to (4.5) 
exist globally in time. If the eigenvalues satisfy the instability index count = k^ = k^ = (see (4.8)), then the 
wave is orbitally stable. In other words, for each e > there is a 6 > such that if 

\\u(0)-U\\x + \\dMO)-T^,mx,<^, 

then 

sup inf (||M(t) - T{g)U\\x + \\dtu{t) - T{g)T^U\\^^ ) < e. 
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4.2. Example: periodic waves to the "good" Boussinesq equation 



The generalized "good" Boussinesq equation (gB) is of the form 

dfu + dl{dlu-u+f{u)) = 0, 



(4.9) 



where / : IR ]R is smooth. In traveling coordinates, i.e., ^ = x - ct with c e (-1,1), the gB can be rewritten 
as 

dju - 2cdl^u + dl{dlu-{\ - c^)u + f(u)) = 0. (4.10) 

The interest will be on solutions to (4.10) which are 2L-periodic in ^, i.e., + 2L, t) = u{E,, t). 

In order to study the existence, spectral, and orbital stability problems, it is convenient to recast the 
gB (4.9) in a Hamiltonian formulation similar to that of (4.2). Herein this task will be accomplished via a 
trick presented in [4]. The evolution is considered to take place on the space Lpg^[-L,+L], i.e., the space of 
square-integrable functions which are 2L-periodic in E,. The inner-product is the standard one, i.e.. 



{f>g) 



dx. 



It is straightforward to check that the original gB (4.9) is equivalent to the system 

dfU = JH'{u), u = (u,vj^, 

where d^v = d^u, 



(4.11) 



dx. 



Here F'{u) = f{u). Note that the above formulation of H is consistent with the formulation of the previous 
section, i.e., 

'dx. 



1 



n(u) = H{u)+-{v,v), m 



while the skew-symmetric operator J' no longer has the property of having a bounded inverse. The system 
is invariant under spatial translation, i.e., T(co)u(x,t) = u{x+ co,t). Consequently, upon using the fact that 
on ker(o'3,)-'- it is true that 

I). 

the conserved quantity associated with the spatial translation is given by 

Q(U)={U,V) (=i^f<M,M)), 

and the Lagrangian for the system is 

A(u) = n{u) + cQ{U) =^ A'(u 
In conclusion, the system to be studied is 



H'{u) + cv 
cu + v 



dtu = JA'(u) 



(4.12) 



which is equivalent to dfU = H'iu) in traveling coordinates ^ = x - ct. 

First consider the existence problem. Since ker(o'^) = spanjl), for real-valued parameters a,b the prob- 
lem is 

- d'!^u + u - f {u) + cv = -a, V + cu = b (4.13) 

which is equivalent to 



-dlu + {l-c^)u-f(u) = -{a+cb). 



(4.14) 
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Figure 1: (color online) The criteria that the potential V{U,a,c) must satisfy relative to the energy E 
in order for there to exist spatially periodic solutions. Here the energy was chosen so that there exist 
two distinct periodic solutions. Increasing the energy past a threshold £*, for which there are two 
homoclinic orbits, yields that the two solutions merge into one periodic solution. 



This is a well-studied problem. In order to use the desired geometric formulation of Bronski et al. [6], it 
will first be necessary to rescale the wave-speed via 

c:=l-c^ =^ c = c+:= +Vl - c, (4.15) 

so that (4.14) can be rewritten as 

-dlu + cu- f(u) = -(a + ch). (4.16) 

Note that in (4.16) the wave-speed c is either of c+. Without loss of generality assume that h = Q. A periodic 
steady-state, say U , will be a solution to the ODE 

dlU-cU + f{U) = a; (4.17) 

hence, the solution will naturally depend on the parameters a and c. If one sets 



^{d^Uf + V(U,a,c), V(U,a,c) := -aU-^( 



then under the assumption that E, a, and c are chosen so that 

(a) E = V{U,a,c) has (at least) two real roots U+ with U_ < 

(b) V{U,a,c) <EforU_<U <U+ 

(see Figure 1) there will be a periodic solution with period 2L, where 

dLf 



j_ r 



^^2Ju_ ^E-V(U,a,c) 

As it will be seen, in particular examples the spectral stability of the 2L-periodic solution will naturally 
depend upon the parameters a,c,E. The dependence of the solution on these parameters will be implicit in 
all that follows. 

Now consider the stability problem. Let the 2L-periodic wave found in (4.16) be denoted by m = [/. 
Recalling that we set b = 0, from (4.13) the v-component of the wave is given by v = -cU, where from (4.15) 
one has c = c+ can be of either sign for a fixed value of c. The steady-state solution for (4.12) is then given 
by 

U 



u = U-. 

\ -CU 

Under the mapping 

u = U + v (4.18) 



Instability index theory for quadratic pencils 



14 



the system (4.12) becomes 

dfV = JA'{U+v) 



The evolution problem must now be considered on a space for which J' has bounded inverse, i.e., on the 



space of mean-zero functions. Let TIq : Lpgj[-L,+L] Hq be the self-adjoint projection operator 



now = M-^(M,i>; 

in other words, llo is the orthogonal projection onto ker(o'j)-'-. Here 

Ho := {u e L^^,[-L,-hL] : <m,1) = 0]=ker{d^)^. 

When writing Hoi' it will be implicitly assumed that Ho is being applied to each component of v. In [8, 
Section 2] it is shown that the proper evolution equation to consider is 

d,v = jnoA'(U + v), v(0)=Vo, (4.19) 

where ngi'o = implies that TlQv(t) = v(t) for all f > 0. In other words, (4.19) describes the evolution of 
mean-zero perturbations of the underlying wave. Since the evolution occurs on Hq x Hq, and d^^ : Hq Hq 
has bounded inverse, in this formulation the operator J now has bounded inverse. 
First consider the spectral stability problem. The linearized eigenvalue problem 

\v = jnQA"(U)v, Uqv = V 

can be rewritten as 

dxiriQH" {U)nQU + cv] = Xv, dy\cu + v] = Am, 
where u,v eHq. Differentiating the first equation yields 

dlnoT-L"{U)nou + cd^id^v) = Xdy,v, 
and substituting the second equation into the first and simplifying gives the quadratic pencil problem 

[a^ - 2cA^;, + dl(UQ(-n"(U) + c^)Uq)\ u = 0. 
Since u e Hq, one can write v = d~^u e Hq, so that the pencil becomes 

d,[A^ - 2cxd, - d,(nQ{n"(U) - c^)nQ)d,]v = o. 

Since has bounded inverse, upon setting £2 to be the well-understood self-adjoint Hill operator 

£^=-dl + c-f'{U{x)), 

the pencil problem to be studied is 

[a^ - 2cXd^ - d^{nQC2^Q)d^'\ v = 0, veHQ. 

Note that in the notation of the previous section, 

C=I, B = -2cd^, A = -d^(nQC2nQ)d^. (4.20) 

Before proceeding with the spectral analysis, the assumptions on the operators given in Theorem 3.1 
must be verified. First consider ker(^). Since £2{d^U) = 0, it is true that 

+L 



£2nQ-d^(U-U) = 0, U=^j^U(x)dx. 
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In other words, U - U e ker(^). In order for ker(^) to have another linearly independent element, it must 
be the case that £2^(1) e Hq. It will be henceforth assumed that no other element in the kernel exists, i.e., 

(£21(1),!) ^0, (4.21) 

so that 

ker(y4) = span{Lf-I7}. 

Letting 

P4 : Ho span! U - U], Pj- : Hq span{ U-U]^cHq 
be orthogonal projections, it must be checked that 

(PA^Pir'PiBP^, (PiAP^)-'PiCP^ 

are compact operators. This immediately follows from the fact that {P^AP^)~^ is compact, and both B 
and C are differentiable operators of lesser order than A. Together, the above considerations verify the 
hypothesis of Lemma 2.1. 

Next, we must verify that the operators A, B, and C are 5-compact for some compact operator S. For 
each a > 0, define the operator 

acting on L^^^([-L,+L]). It is clear that S^^ is a compact self-adjoint operator on Hq for each a > 0; further- 
more, it is true that the space 

Xs^={ueHo:{no{dl + lf"nou,u)<oo] 

is dense for any a > 0. Now, clearly the operator S^^CS^^ = S^^ is compact for any a > 0. Regarding the 
operator B, it is easy to see that S'^d^S'"^ will be compact as long as 1/4 < a. Finally, the operator S'^^AS'^ 
will have a compact resolvent as long as < a < 1. In conclusion, as long as 1/4 < a < 1, the operators will 
be 5„-compact, thus verifying hypothesis (a) and (b) of Theorem 3.1. 

From the skew-symmetry of the operator and the fact that U is 2L-periodic it is clear that 

Slker(^) = -2C< U-U,dAU- U)) = 0. 

Provided that (I - 4c^d^A~^ d^)\ 

kei(A} is invertible then, a direct application of Theorem 3.1, using the 
explicit form of the operators given in (4.20) and noting that C = I is clearly a positive definite operator, 
implies that the index count satisfies 

k, + k, + k- = n(A) - n((I - 4c2^^^-i5^)|kerM,). 

In order to compute n(^), first note that 

{u,Au) = {u,-d^(T\QC2^Q)d^u) = {d ,.u,IlQC2TlQ(d ^u)). 

Thus, upon using the fact that d^-.H^^ Hq has a bounded inverse it is clear that 

n(A)=n(noC2no). 

Regarding the quantity on the right, it was shown in [8, equation (2.25)] that if the inequality of (4.21) 
holds, then 

n(no£2no)=n(/:2)-n«£2i(l),l)). 
Consequently, it can now be said that 

n{A)=n{C2)-n{{C-2Hni)), (4.22) 

so that the index count satisfies 

K + k, + k- = n{C2) - n{{C-2^ ( 1 ), 1 )) - n((I - 4.C^d^A-' 5 Jlker(yl))- 
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Recalling that = 1 - c, the index count is complete once the scalar 

(1-4(1 -C)5^^-I5,)|ker(yt) 

is computed. From (4.20) one has that 

I - 4(1 - c)d,A-'d,=I + 4(1 - c)d, ■ d-^noCjUor^d-^ ■ 

= 1 + 4(1 -cxnoAno)-' : 

the second line follows from the fact that is invertible on Hq. Using the characterization of ker(^), it is 
then seen that 

(I - 4( 1 - c)d,A-' 5;,)lker(.4) = {u-u,u-u) + Mi- c){{noC2no)-' {u-u),u- u). 

Finally, in the study of the orbital stability of periodic waves for the generalized Korteweg-de Vries equation 
(gKdV) it was shown in [8, Section 3] that 



<(no/:2nor(u-u),t/-t/) = D„Kdv, d, 



gKdV 



Lemma 4.2. Consider the quadratic pencil (4.19). If {£2^(1), I) * 0, then the stability index is given by 



kr + kr + k^= n(C2) - -^({C2\l),l)) - n({U - U, U - 17) + 4(1 - c)D, 



gKdV; 



■where 



D. 



gKdV 



and the parameter c is related to the original wave-speed c via c^ = 1 - c. 

Remark 4.3. It was shown in [8, Theorem 2.6] that the stability index for the gKdV is given by 

k, + kr + k, = n(C2) - n({C-2'(n 1 )) - n(DgKdv); 

hence, when studying the spectrum for periodic waves to gB and gKdV there is an intimate connection in 
the indices for the two problems. If Dr,Y,dv < 0, then for 



1 > c > 1 + 



{U-U,U-U) 



4D, 



gKdV 



the stability index for the gB is exactly the same as for the gKdV. Otherwise, there is precisely one more 
eigenvalue which is counted by the index. On the other hand, if DgKdv > 0, then the index for the quadratic 
pencil is exactly that for the gKdV equation. 

Remark 4.4. There is a geometric interpretation associated with the quantity DgKdv- The interested reader 
should consult [6] for more details. 

Remark 4.5. In Hakkaev et al. [15] the spectral stability problem was considered under the additional as- 
sumption that n(£2) = 1 ■ Furthermore, while it is not explicitly stated, they further assume that (£2^(1),!) > 
0, so that (in this paper's notation) the index becomes 

k, + kr + k^ = l-n{{U-U,U-U) + A{l- c)D^Kdv)- 

The instability criterion in that paper follows from the fact that kr, A;^. must be even integers, so that kr = 
kr = 0, with 



|0, 



c < c*. 
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^1 + 1 



where 

{U-U,U-U) 
c* = l + ^ — ^ 

4DgKdv 

It is not clear if in that paper an expUcit connection is shown between the gB and the gKdV. 

Now consider the orbital stability problem. The local global well-posedness problem has been studied 
in, e.g., [3, 10, 11], and it will henceforth be assumed that the problem can be solved (at least) locally. 
Depending on the growth rate of the nonlinearity, this implies that the initial data for (4.19) satisfies, e.g., 
vi(0) e Hpgj[-L,+L], and V2{0) e H~^^[-L,+L], where the norm for the latter space is given by 

The form of the Lagrangian in (4.12) makes clear that in order to control the nonlinear terms the proper 
space in which to work is Hpg^[-L,+L] x Lpg^[-L,+L]. It will be further assumed that the hypothesis leading 
to Lemma 4.2 hold, and that the spectral problem has zero instability index, i.e., = = = 0. As was 
seen in [8, Section 2.4], this is sufficient in order to conclude that the wave is orbitally stable with respect 
to the evolution defined by (4.19). 

Proposition 4.6. Suppose that the IVP for (4.19) is locally vjell-posed. Further suppose that in addition to what 
is required for a unique local solution to exist, the mean-free perturbative initial data for the system (4. 1 9) satisfies 
v(0) e Hpgj[-L, +L] X Lpgj.[-L, +L]. If the spectral problem satisfies k^ = k^. = k^ = 0, then the underlying wave is 
orbitally stable. In other words, for each e > there is a 6 > such that for (4.19), 

\\u{0)-U\\„i ^,2 <3 =^ sup inf ||u(t)-T(a;)[7|Li ^,2 < e. 

Note that for the original system (4.9) the requirement on the initial data is 

u(0) = U + ViiO), dtu{0) = -cd^U + d^V2{0), 

where each component Vj(0) has zero mean. A natural question is then: what happens if the initial pertur- 
bation is not mean-free? In this case, we now argue for orbital stability with respect to a nearby periodic 
traveling wave of (4.19). For related arguments in the contexts of other nonlinear dispersive equations, see 
the work of Haragu§ and Gallay [16] on the nonlinear Schrodinger equation as well as the works of Johnson 
[18, 19] on generalized KdV and BBM models, respectively. 

To begin, we make a few comments regarding the conserved quantities of (4.11). As discussed above, 
even though we set = in the analysis, the periodic traveling wave solutions of (4. 1 9), which are solutions 
to the ODE (4.14), form a five-parameter family of solutions of the form 

(x, t) = u{x- ct + £,;a,E,c, b) 

where £. eM and {a,E,c,b) belong to some open set Q e ]R*. Furthermore, the evolution equation (4.11) 
admits the following two conserved quantities: the momentum (charge) 

P(w) := J" WiW2dx, w := (11^1,^2) e Hp„[-L,-i-L] X Lpgj.[-L,-i-L], 

arising from the translation invariance of (4.19), and the casimirs 

A^i(w):=J" widx, M2{it^-='j W2dx, w:= {wi,W2) e IIp^^[-L,+L]xLp^^[-L,+L], 

arising from the fact that ker( ^7) is non-trivial. Notice that V and M are smooth functionals on Hpg^[-L,+L]x 
Lpgj[-L,+L] and that, when restricted to the manifold of traveling wave solutions of (4.19) the functionals 
jCii, M.2, and V reduce to 

Mi{a,E,c,b):= { u{x;a,E,c,b)dx, M2(a,E,c,b) := cMi{a,E,c,b)-bT, 



Instability index theory for quadratic pencils 



18 



and 



P{a,E,c,l 



-P{a, E, c, b) + bMi{a, E, c, b), 



where here T = T{a,E,c,b){= 21) denotes the period of the wave and P{a,E,c,b) := c u{x;a,E,c,b)^ Ax. 

Now, consider the case where the means of Vj{Q) are small, but non-zero. Using the geometric formalism 
of Bronski et.al (see [5, 6]) we have the following key lemma. 

Lemma 4.7. With the notation as above, we the equality 



ker(^) 



[ Te 


Mi,E 


Pe 


M-2,E 






Pa 


M2,a 


T 


Ml,, 


Pc 


M2,c 


> Tb 




Pb 


M2,b 



Proof: To compute the left hand side, define the function 



</)2(x) := det 



M 



l,a 



"c Tc Mi^c ) 



M 



l,a 



Te Mi,£ 



{U-U). 



□ 



and satisfies 

no£2no</>2 = no>C2^2 = 2cdet 
The stated equality now follows by directly calculating 

(1-4(1 -c)5,^-i5,)lke,(^, = (noi/,(i + 4c2(no/:2nor')noL/) 

and comparing to the right hand side of the above equality. 
From Lemma 4.7 and the assumption that 

(l-4(l-c)5,,^-l5^)|kerM) 

is nonsingular at the underlying wave U, corresponding say to {a,E,c,b) = (aQ,EQ,CQ,bQ), implies that the 
map 

M^B(a, E, c,b)^[ T{a, E, c, b), {a, E, c, b), P{a, E, c, b),M2{a, E, c, &)) e 
is a local diffeomorphism from a neighborhood of (aQ,EQ,CQ,bQ) onto a neighborhood of the point 

(T,Mi,P,M2){aG,Po,cM- 

It follows that we can find a curve [0, 1 ] 3 s {a{s), E(s), c(s), b(s)) e with (a(0),E(0), c(0), b(0)) = (0, 0, 0, 0) 
such that for each s e [0, 1] the function 

u{x;s) = u {x;aQ + fl(s),£o + E{s),Cq + c{s), b^ + b{s)) 

is a T = T(aQ,EQ,CQ,bQ)-periodic traveling wave solution of (4.9) and that, moreover the endpoint condition 

Mj (flo + a{l),Eo + £(l),Co + c{l),bo + b{l)) = Mj (u(0) + 1^(0)), ; = 1,2 

P{ao + fl(l),£o + E{l),co + c{l),bo + b{l)) = r{u{0) + v{0)) 

and growth constraint 

sup \^a^s),E(s),c(s),b(s))\^4<\\vm\^^^^^^_^^^^^^^^^^. 



sup |(fl(s),£(s),c(s),fc(s))|iR4 <||i^(0)||h1 (_i„L)x4„(-L,L) 
se(0,l) 



are satisfied. Assuming ||i'(0)||^i is sufficiently small, it follows that the wave «(•, 1) is nonlinearly 

orbitally stable in the sense described in Proposition 4.6, which, by the triangle inequality, implies orbital 
stability of U to initial perturbations i'(O) with nonzero, but sufficiently small, mean. Since Mj and V 
are continuous in the Hpgj. x Lpg^ topology, it follows that we have orbital stability in the standard sense 
without the restriction to mean-free initial data in (4.19). This observation yields the following extension 
of Proposition 4.6 
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Theorem 4.8. Suppose that the IVP for (4.19) is locally well-posed. Further, suppose that in addition to what 
is required for a unique local solution to exist, the perturbative initial data for the system (4.19) satisfies v{0) e 
Hpgj[-L, +L] xLpgj.[-L, +L]. If the spectral problem satisfies = kc = kj = 0, then the underlying wave is orbitally 
stable, i.e., for each e > there exists a 6> such that for (4.19) we have 

\\u{0)-U\\„i <6 =^ sup inf ||M(t)-T(a))t7|Li ^,2 < e. 

In the next sections, we utiUze the above expUcit connection of the stabiUty problems for gB and gKdV 
type equations to make several comments regarding the stability of periodic waves in the gB equation. 

4.2.1. Case study: power law nonlinearity 

The stability indices for the gKdV equation were computed in Bronski et al. [6] for the case that f{u) = mP+^ 
for some p > 1 and |c| < 1. Note via (4.15) that this implies \c+\ < 1. While we will not do it here, the gB 
can be rescaled so that the influence of c is removed from the steady-state problem. This independence is 
reflected in the existence diagrams. As in the analysis leading to the statement of Lemma 4.2, it will be 
assumed here that b = 0. 

First suppose that p = 1, which corresponds to the classical gB equation. In [6, Section 5.1] it is shown 
that all periodic waves in this model satisfy 

n(£2) = l, n«/:2i(l),l) = 0, n(DgKdv) = l. 

Thus, via Lemma 4.2 it is true that for a given {a,E) there is a critical positive wave-speed c* ^ < 1 such that 
fcr =k^ = with 

[0, C<Ca,E- 

Returning to the original wavespeed c, it follows that for any periodic traveling wave solution of the clas- 
sical gB equation there exists a range of wavespeeds (1 - c^f)^^^ < k| < 1 for which the wave is nonlin- 
early (orbitally) stable, while it is unstable spectrally unstable to perturbations with the same period if 
|c| < (1 - Cflg)^^^- This is consistent with the result of [15, Theorem 2], where c^ e is explicitly given when 
fl = 0. In that paper the case of nonzero a was not considered and only spectral instability for |c| < (1-c* f)^''^ 
was verified. Here, our calculations compliment this result by verifying that waves traveling with speed 
greater than (1 - f)^^^ are by Proposition 4.6 indeed nonlinearly stable. 

For examples which are not covered in Hakkaev et al. [15], e.g., when it is possible for n(£2) ^ 2, first 
consider the problem when p = 2. The table below, which corresponds to Figure 2, can be derived from [6, 
Section 5.2]: 



Region 


n(£2) 


n«/:-i(l),l)) 


n(DgKdv) 


(a) 


1 





1 


(b) 


2 





1 


(c) 


2 


1 





(d) 


2 


1 


1 


(e) 


1 





1 



From the theoretical result in Lemma 4.2 it will be the case that in that in regions (a), (d), and (e) there 
will exist a < <;„ £ < 1 such that k^ = \ for c > <;„ £ and k^ = Q otherwise; furthermore, it is always true that 
fcr = fcc = in these regions. In region (c) it will be the case that k^ = \ for all c with the other two indices 
being zero. Finally, in region (b) there will exist a < c„ £ < 1 such that k^ = \ for -1 < c < c„ £ with the 
other two indices being zero, while for c > c„ £ all that can be said is that A:^ + + A:^ = 2. Notice, however, 
that by parity we see for speeds c > c* ^ in region (b) we have k^ = and k^ + k^ = 2, which allows the 
possibility that some waves may still be spectrally stable in this region with k^ = 2 or that some waves may 
be spectrally unstable to perturbations with the same period with k^ = 2: such a situation is precluded in 
the well-studied solitary wave theory. 
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-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 



Figure 2: (color online) The configuration space in the fl£-plane when p = 2, b = 0, and |c| < 1 
fixed (see [6, Figure 3]). The swallowtail figure divides the plane into regions containing 0, 1, and 2 
periodic solutions. In region (a) there are two solutions, while all of the other marked regions have 
one solution. In the unmarked region there are no periodic solutions. The quantities used in the 
stability calculation are given in an accompanying table. 

Remark 4.9. In [15, Theorem 1] it is shown that for one of the two solutions in region (a) of Figure 2 with 
fl = the index satisfies = k^ = with k^ = I for c > c^ f, and k^ = for c < £. Furthermore, for this 
solution the constant ^ is explicitly given when a = 0. Although they do not consider the case in their 
paper, the same result holds in region (e). The parameter region which is outside their theory comprises 
the union of (b), (c), and (d). 

The below table for p = 4 which corresponds to Figure 3 can be derived from [6, Section 5.3]: 



Region 


n(£2) 




n(DgKdv) 


(a) 


1 





1 


(a') 


1 





1 


(b) 


1 








(c) 


2 


1 





(d) 


2 





1 



From the theoretical result in Lemma 4.2 it will be the case that in that in regions (a) and (a') there will exist 
a < c* £ < 1 such that k,. = I for c > c^ g and k^ = otherwise; furthermore, it is always true that k^ = k^ = 0. 
In regions (b) and (c) it will be the case that k^ = I for all c with the other two indices being zero. Finally, 
in region (d) there will exist a < c„ £ < 1 such that = 1 for c < £ with the other two indices being zero, 
while for c > £ all that can be said is that k^ + k^ + k^. = 2. 

4.2.2. Case study, continued: solitary wave and equilibrium solution limits 

In this final section, we make some comments regarding the stability of periodic traveling wave solutions 
of (4.9) which are either near the solitary wave or near an equilibrium (constant) solution. Throughout this 
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-1.0 -0.5 0.0 0.5 1.0 



Figure 3: (color online) The configuration space in the fl£-plane when p = 4 and |c| < 1 (see [6, 
Figure 4]). The swallowtail figure divides the plane into regions containing 0,1, and 2 periodic 
solutions. In region (a) there are two solutions, while all of the other marked regions have one 
solution. In the unmarked region there are no periodic solutions. The quantities used in the stability 
calculation are given in an accompanying table. 



section, we continue to consider (4.9) with power law nonlinearity f{u) = mP"""^ for some p > 1. In this case, 
we have from (4.14) that the profile U satisfies the ODE 

dlu = (I - c^)u - uP^\ 

where, for simplicity, we are restricting our discussion to those waves with a = b = 0^. This equation is 
clearly Hamiltonian, and has critical points (m, ^^.m) = (0, 0), corresponding to a saddle point, and {u,d^u) = 
((1 - c^)^^P,0)^, corresponding to a nonlinear center. Further, for a fixed wavespeed c e (-1, 1), in the two- 
dimensional (u,d^u) phase plane the nonlinear center ((1 -c^)^^P, 0) is surrounded by a one parameter 
family of periodic orbits, which are in turn bounded by an orbit which is homoclinic to the saddle point 
(0,0). These periodic orbits can be parameterized by the ODE energy £ determined from the defining 
relation 

2^ ' 2 p + 2 

The period T(E,c) of these waves inside the homoclinic orbit satisfies 

271 

lim T{E,c) = +oo, lim T{E,c 



where E*{c) is the ODE energy level associated with the equilibrium point ((1 - c^)^^'', 0). Below, we con- 
sider the stability of the periodic traveling wave solutions of (4.9) in both the distinguished limits E 0~, 



^As we will see below, this is a natural restriction when considering the limiting case to a solitary wave asymptotic to zero 



X — > +00. 

2 



Notice when p is an even integer, the point (u.d^u) = (-(1 -c^)^^P,0) is also a critical point. In this discussion, we ignore this 
additional critical point, noting that any conclusions for periodic waves emerging from the ((1 - c^)^^P ,0) critical point hold also for 
those emerging from the (-(1 - c^)^^'', 0) critical point. For general p > 1, the governing ODE does not admit such negative solutions. 
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corresponding to the solitary wave limit, and E £*(c)"'", associated with small amplitude periodic wave 
trains. 

We begin by considering the stability of the periodic traveling waves of (4.9) in the solitary wave limit. 
When considering solitary wave solutions which decay to zero as x ^ +00 we obtain a two-parameter 
family of solutions parameterized by wavespeed c and spatial translation, which corresponds to a one- 
parameter family of homoclinic orbits in the two-dimensional phase space of (4.17) (notice the boundary 
conditions in this case require a = E = b = 0). Keeping throughout a = b = and fixing the wavespeed c e 
(-1, 1), by the above considerations there exists, up to translations, a one parameter family of "large period" 
periodic traveling wave solutions of (4.9) parameterized by the ODE energy E. Furthermore, for fixed 
c these periodic waves approach locally uniformly on IR an appropriate translate of the limiting solitary 
wave as £ ^ E~. The stability of the limiting solitary waves was investigated by Bona and Sachs [4], where 
they applied the abstract theory of Grillakis et al. [12] to obtain nonlinear stability of the solitary wave 
when 1 < p < 4 and p/4 < < 1. This stability theory was later complimented by Liu [28], obtaining 
nonlinear instability if either < p/4 or p > 4. Next, we show that this phenomenon is observed again for 
the "nearby" periodic traveling wave solutions of (4.9). For such periodic waves, it is easy to see that 

n(/:2)-n«£2^(l),l)) = 1, 0<-£<sl, 

and, furthermore. 



lim DgKdv 



, , =-A(c,p)(4-p) 

{a,E,c,b)={0,E,c,0) 



for some positive constant A(c,p) > (for details, see the asymptotic analysis in [5, Section 3.2]). It follows 
that for alP 1 < p < 4 there exists a critical wavespeed c = c*{p,E) such that the periodic traveling wave 
u{-;a = 0,E,c,b = 0) with < -£ 1 is orbitally stable for c*(p,E) < jcj < 1 and spectrally unstable with 
fcj. = 1 for jcj < c*(p,E), while for p > 4 all such long-period waves waves are spectrally unstable with fc,. = 1. 
This is consistent with the solitary wave orbital stability/instability results of [4] and [28]. 

Finally, continuing to restrict to a = b = 0, we consider the stability of small amplitude periodic traveling 
waves of (4.9) associated to those periodic orbits of the profile ODE near the nonlinear center ((1 -c^)^^P,0). 
Fixing the wavespeed c e (-1,1), it follows by basic asymptotic analysis near the equilibrium solution 
M=(l-c2)i/Pthat 

n(£2)-n«/:2i(l),l)) = l 
and, furthermore, that at the nonlinear center we have 

DgKdv(fl = 0,E'{c),c,b = 0) = -Q(c,p)y" ((1 -c2)i/P, 0,1 -c^)"''''^ = -n(c,p)((l - c2)p)"''''^ 

for some positive constant Q(co,p) > (see [18, Section 5] for details). It follows that in a neighborhood of 
the equilibrium solution m = (1 - c^)^^P there exists a critical wavespeed c(p,E) such that all nearby small- 
amplitude periodic traveling wave solutions of (4.9) of the form u{-;a = 0,E,c,b = 0) with Eq < E <^ I are 
orbitally stable for c*{p,E) < jcj < 1, and are spectrally unstable to perturbations with = \ for < jcj < 
c*{p,E). In particular, the equilibrium solution itself is orbitally stable for all jcj e (-1,1), implying that 
c*{p,E) 0* as E ^ (^0) ' approaches the equilibrium solution. Note this is consistent with the 

numerical calculations of c*{p,E) presented in [15] in the cases f{u) = and f{u) = u^. 
Remark 4.10. It is interesting to note that when p > 4 the waves with a = b = undergo a transition to 
instability as one moves from a neighborhood of the equilibrium solution w s (1 - c^)^^P to a neighborhood 
of the limiting solitary wave. 
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